The invention relates in general to machine vision and in particular to devices for recognizing human gestures.
Several approaches have been made to perform recognition of human motion, with emphasis on real-time computation. The most frequently used methodologies for recognition of body motion and dynamic gestures are based on the analysis of temporal trajectories of motion parameters, hidden Markov models and state-space models, and static-activity templates. Other conventional techniques have attempted to represent motion by collecting optical flow over the image or region of interest throughout the sequence, although this is computationally expensive and often not robust. Another conventional technique combines several successive layers of image frames of a moving person in a single template. This single template represents temporal history of a motion, allowing a match of actual imagery to a memorized template to produce recognition of a motion gesture.
The above conventional techniques have been conducted in controlled laboratory environments, with fixed lighting and constant distance to the subject. Actual field conditions will present external stimuli, resulting in difficulties in recognition. Thus, it is apparent that different approaches are required for real-life applications outside of a laboratory, such as the flight deck of an aircraft carrier.
Flight deck operations are a “dance of chaos” with steam, constant motion, and crowding. These operations are conducted during day or night, rain or snow, when visibility is extremely poor. Moreover, fleet operations are continually subject to reduced manning and resistance to change. It is desired that, in these kinds of conditions, Unmanned Combat Air Vehicles (UCAV) shall be launched from the flight decks of aircraft carriers.
To launch a UCAV from a flight deck, the UCAV must be controlled during taxiing, before takeoff and after landing. Simply hooking a tow tractor to the aircraft has been considered, but was deemed too slow, especially since the aircraft are required to recover every 45 seconds and need to taxi out of the landing area for the next aircraft. Alternatively, providing the aircraft director/controller with a joystick to control the aircraft would tax his/her workload with an additional process, and would negatively impact training and operations.
Further, if the UCAV is to be controlled on deck using a joystick, the UCAV would necessarily be controlled via radio (RF) link. However, an RF link from a control device to the UCAV is undesirable because of the EMI (electromagnetic interference) intensive environment on the flight deck. Another alternative is a tethered connection, using a control device physically tethered to the UCAV. However, such a tethered connection may be potentially unsafe for the personnel on the deck during high activity periods.
Like manned aircraft, the UCAVs taxi before launch, after recoveries or during re-spotting. In the case of manned aircraft, flight deck controllers signal directions to pilots for taxiing the aircraft around the deck or airfield. It would be desirable if these signals were used to develop an automatic taxiing system for unmanned aircraft as well. If such a system were developed, it would enable a seamless transition of UCAVs into naval aviation.
It is an object of the invention to provide a method and apparatus for the recognition of human gestures by a machine.
It is an object of the invention to provide a method and apparatus for the recognition of human gestures by a machine wherein the method requires much less computation than known methods.
One aspect of the invention is a method of identifying a human gesture comprising providing a time sequence of data related to the human gesture; transforming the time sequence of data into waveforms; extracting features from the waveforms; and identifying the human gesture based on the extracted features.
In one embodiment, the providing step includes providing a time sequence of pixel images of the human gesture using at least one video camera. In another embodiment, the providing step includes providing the data using accelerometers attached to the human.
The extracting step may include extracting static features from the waveforms and may further include extracting dynamic features from the waveforms. In one embodiment, the extracting step includes extracting hand position, phase and frequency from a right hand waveform and a left hand waveform.
The identifying step may include identifying the human gesture by comparing one or more of the hand position, phase and frequency from the right hand waveform and the hand position, phase and frequency from the left hand waveform to at least one rule that describes the human gesture.
The method may further comprise classifying hand positions into vertical and horizontal ranges. The vertical and horizontal ranges are preferably defined in terms of a characteristic length of a human performing the gesture. A preferred characteristic length is the distance from a cranial point to a torso point.
In one embodiment, hand position in the vertical range is determined using a fuzzy logic method.
Another aspect of the invention is a computer readable medium containing a computer program for performing a method of identifying a human gesture, the method comprising transforming a time sequence of data related to the human gesture into waveforms; extracting features from the waveforms; and identifying the human gesture based on the extracted features.
A further aspect of the invention is an apparatus for identifying a human gesture comprising means for providing a time sequence of data related to the human gesture; means for transforming the time sequence of data into waveforms; means for extracting features from the waveforms; and means for identifying the human gesture based on the extracted features.
The means for providing may comprise, for example, accelerometers or at least one video camera. The means for transforming, the means for extracting and the means for identifying may comprise at least one computer.
The invention will be better understood, and further objects, features, and advantages thereof will become more apparent from the following description of the preferred embodiments, taken in conjunction with the accompanying drawings.